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Emotional/behavioral disturbance (EBD) is characterized by a range of behaviors that adversely affect 
a child’s academic performance and cannot be explained by other sensory or health impairments. Al- 
though research has clearly demonstrated that children and youth with EBD tend to exhibit high rates 
of problem behavior, research on the characteristics of their academic performance has been less clear. 
This article reports the results of a meta-analysis of the academic status of students with EBD. The over- 
all effect size was -.64, which indicated that students with EBD had significant deficits in academic 
achievement. An examination of moderators (subject area, setting, and age) indicated that students with 
EBD performed at a significantly lower level than did students without disabilities across academic 
subjects and settings; greater deficits were not observed in older students with EBD (i.e., those more than 
12 years old), as compared to younger students. Implications and areas for future research are discussed. 


Emotional/behavioral disturbance (EBD) is characterized by 
a range of problems that adversely affect a child’s academic per- 
formance and cannot be explained by intellectual, sensory, or 
other health factors (Individuals with Disabilities Education 
Act, 1997). These behaviors can include a number of internal- 
izing and externalizing characteristics that inhibit a child’s abil- 
ity to build and maintain successful social relationships with 
peers, teachers, and adults. Children and adolescents with EBD 
are overwhelmingly male, behaviorally disruptive, noncompli- 
ant, verbally abusive, and aggressive. Because their behaviors 
are so disruptive and irritating, these children often arouse neg- 
ative feelings in others, alienating schoolmates and adults and 
ultimately robbing these children of the benefits of learning 
opportunities (Kauffman, 2001). Inevitably, these behaviors sig- 
nificantly impair a child’s ability to succeed in school and in 
society. 

Although research has unequivocally demonstrated that 
children and youth with EBD exhibit high rates of problem 
behavior, research on the characteristics of their academic per- 
formance remains uncertain (Kauffman, Cullinan, & Epstein, 
1987). Data on identification, academic outcomes, graduation 
rates, absenteeism, employment status, and criminality among 
children and youth with EBD has suggested that educating them 
is a complex, confusing, and often daunting task for educators, 
related services personnel, and family members alike (Smith & 
Coutinho, 1997). More specifically, the preponderance of stud- 
ies on academic performance have indicated that students with 
EBD perform 1 to 2 years below grade level (Trout, Nordness, 


Pierce, & Epstein, 2003), with academic difficulties emerging at 
an early age and persisting throughout their schooling (Cou- 
tinho, 1986; Rosenblatt & Rosenblatt, 1999; Wagner, 1995). 
This should be expected, because current criteria for identifi- 
cation of EBD require that students show a deficit in academic 
achievement. Compared to peers from other high-incidence 
disability groups, children with EBD evince lower reading and 
math scores, lower graduation rates, and higher rates of course 
failure and grade retention, and are less likely to attend postsec- 
ondary school (Kauffman, 2001; Wagner, 1995). In 1998-1999, 
50.6% of students age 14 and older with EBD had dropped 
out of high school (U.S. Department of Education, 2001). Eur- 
thermore, it has been estimated that 70% of students with EBD 
will be arrested within 3 years of leaving school, continuing 
a pervasive pattern of failure that becomes difficult to correct 
(U.S. Department of Health and Human Services, 1999). 

Low academic performance and maladaptive behavior 
patterns are highly related. Evidence suggests that a reciprocal 
relationship between school failure and social failure emerges 
early in life (Brier, 1995; Kauffman, 2001), although the mech- 
anism of the relationship is unknown. The prevalence of aca- 
demic difficulties among children with EBD is uncertain. 
Ruhl and Berlinghoff (1992) suggested that between 33% and 
81% of children with behavioral disorders have academic dif- 
ficulties. A causal relationship between behavioral problems 
and academic underachievement, however, has yet to be de- 
termined (Hinshaw, 1992). Nonetheless, researchers have dem- 
onstrated that academic failure is one of the most powerful 
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predictors of problem behavior and social failure (Maguin & 
Loeber, 1996; Morrison & D’Incau, 1997). Conversely, re- 
searchers have also demonstrated that academic success is as- 
sociated with a decrease in problem behavior (Gottfredson, 
Gottfredson, & Skroban, 1996). 

To further our understanding of the academic status of 
students with EBD, researchers have traditionally relied on 
conventional narrative reviews of the literature (e.g., Epstein, 
Kinder, & Bursuck, 1989; Trout et al., 2003). These reviews 
have found that students with EBD perform significantly below 
norms on standardized achievement tests (e.g., Coutinho, 
1986) and achieve lower scores in mathematics than in read- 
ing (e.g., Schroeder, 1965; Stone & Rowley, 1964) and that 
older children with EBD are farther behind their same-age 
peers than are younger children with EBD (Couthino, 1986). 
Although these qualitative reviews have presented compelling 
evidence of the academic deficits experienced by children with 
EBD, they do not provide precise quantitative estimates of the 
magnitude of these deficits. 

A quantitative review of the research on the academic per- 
formance of students with EBD compared to same-age, non- 
disabled peers or norm groups would extend our knowledge 
in this important area. Previous investigations have focused 
on grade-equivalent scores (e.g., Rosenblatt & Rosenblatt, 
1999; Scruggs & Matropieri, 1986); however, these scores do 
not provide information on whether performance is within av- 
erage limits, allow for normative comparisons, or allow for 
comparisons across studies. One statistical method for ana- 
lyzing a body of research to compare group performance is 
meta-analysis. Meta-analytic reviews go beyond traditional 
narrative reviews in the sense that they are more systematic, 
more explicit, and rely on quantitative methods of analysis 
(Rosenthal, 1984). Because of these features, meta-analytic 
reviews are considered to provide more thorough, compre- 
hensive, and precise summative evaluations that entail greater 
objectivity than narrative reviews (Rosenthal, 1984). The pur- 
pose of this study was to use a meta-analytic approach to quan- 
titatively estimate the magnitude of difference in academic 
performance between students with EBD and their same-age, 
nondisabled peers or norm groups. The use of meta-analysis 
allows for a precise estimate of achievement difference, which 
allows for normative comparisons, provides a metric that is 
consistent across age levels, and allows for comparisons across 
studies. Moreover, meta-analysis is consistent with the Amer- 
ican Psychological Association’s guidelines calling for the use 
of effect sizes, which allows for an evaluation of the practical 
significance of differences (as opposed to statistical signifi- 
cance). 

This study focused on the following. Eirst, we hoped to 
determine whether there were differences in academic status 
across age, gender, race, and socioeconomic status. Age is an 
important factor because academic difficulties of children with 
EBD are thought to increase over time (e.g., Coutinho, 1986). 
Gender, also important, is an understudied area. Despite the 
fact that girls constitute approximately 20% of children with 


EBD (U.S. Department of Education, 2001), little is known 
about possible gender differences in academic status. Simi- 
larly, achievement differences among racial/ethnic groups 
(Meece & Kurtz-Costes, 2001) and among individuals from 
different socioeconomic levels (Dixon-Eloyd Johnson, 1998) 
have been well documented and may occur within EBD groups. 
Second, we examined the possibility of differences across 
academic subject areas. Differences in achievement across 
academic subject areas would have direct implications for in- 
struction of students with EBD. Third, we investigated place- 
ment setting as a possible moderator of academic status. Little 
is known about the effects, if any, of setting on academic sta- 
tus of students with EBD. Given the increasing trend towards 
inclusion of students with EBD in less restrictive settings, 
comparisons of academic outcomes across setting is especially 
relevant. Finally, we investigated whether method of iden- 
tification (e.g., school identified, DSM [Diagnostic and Sta- 
tistical Manual of Mental Disorders; American Psychiatric 
Association, 2000] identified) had effects on academic status. 
Students identified via different criteria might differ in sever- 
ity. For example, children identified via DSM criteria might 
exhibit more serious behavior problems and be placed in much 
more restrictive settings. Thus, different identification criteria 
could result in qualitatively different subgroups of children 
who differ in academic achievement. 

Method 

Study Selection Criteria 

In this meta-analysis, we used a five-step literature search strat- 
egy to identify relevant studies published between 1961 and 
2000. First, computer searches of PsychINFO and ERIC were 
conducted. The literature search terms used were emotional 
disturbance, emotional and behavioral disorders, behavior 
disorder, conduct disorder, behavior and academic status, aca- 
demic achievement, reading, math, arithmetic, written expres- 
sion, and academics. Second, manual searches of all issues of 
the Journal of Emotional and Behavioral Disorders and Be- 
havioral Disorders were conducted. Third, we conducted 
ancestral searches of all identified articles. Fourth, articles pre- 
viously reported in reviews of the literature were examined. 
Finally, researchers who had written on the academic status 
of children with EBD were contacted via e-mail and asked 
for assistance in locating further articles. This search strategy 
yielded a total of 205 articles potentially relevant to the meta- 
analysis. 

Two graduate students screened the Method and Results 
sections of all 205 articles, to ensure that the articles included 
data from students with EBD and had reported academic 
achievement as a dependent variable. In the cases (« = 6) where 
a disagreement occurred, a third reader independently assessed 
the article and a decision was made to include or exclude the 
article. This screening resulted in the exclusion of 104 articles 
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that did not specifically relate to the academic status of stu- 
dents with EBD or that did not provide data (e.g., practitioner 
pieces). Thus, 101 articles met the initial search criteria for 
inclusion in the review. 

Inclusion Criteria 

Inclusion criteria were chosen to ensure that the studies sam- 
pled accurately reflected the literature on the academic status 
of children and youth with EBD. Three graduate students read 
all 101 articles obtained during the preliminary search. Eor ar- 
ticles to be included in the literature review, each of the fol- 
lowing conditions must have been met: 

• The study was published in a peer-reviewed 
journal between 1961 and 2000. Only peer- 
reviewed studies were considered, as a means 
of overall quality control (Weisz, Weiss, Han, 
Granger, & Morton, 1995). 

• Participants in the study were identified as EBD 
by one of the following methods: (a) identified 
through the school/individuals with Disabilities 
Education Act (IDEA), (b) identified through 
DSM-IV (or prior versions) with conduct dis- 
order or co-occurrence of EBD and another 
disability (e.g., attention-deficit/hyperactivity 
disorder), (c) currently being served in pro- 
grams for severe behavior problems (e.g., 
psychiatric hospital, university clinic), or 

(d) identified as functioning in the clinical range 
through the use of behavior rating scales. 

• The study included a mean score and standard 
deviation from a standardized test on at least 
one academic content area. 

• The study sample consisted of children and 
youth between 5 and 21 years of age. 

After applying these criteria, an additional 75 studies 
were excluded, leaving 26 studies that met the inclusion cri- 
teria for this meta-analysis. A number of factors resulted in the 
exclusion of the 75 studies: 

1 . Thirty-three studies were eliminated either 
because they were small sample size studies 
or because only grade-equivalent scores were 
used as the dependent measure. In the cases 
where grade-equivalent scores were provided, 
we were unable to derive a score suitable for 
meta-analysis, because scores were reported 
as aggregates across age ranges (e.g., a mean 
grade-equivalent score was reported for an age 
range). 

2. Seventeen studies failed to provide data regard- 
ing specific academic characteristics of the stu- 


dents (e.g., academic status was determined by 
assessing school competence on behavior 
checklists). 

3. Six studies presented subtest scores from IQ 
measures. 

4. Eight studies used Individualized Education 
Program (lEP) goals and objectives as the de- 
pendent variable or reported only narrative data 
(e.g., poor performing, below grade level). 

5. Nine studies did not disaggregate the scores of 
students with EBD from their peers with other 
disabilities. 

6. Two studies reported only correlation between 
academic measures. 

Data Analysis Strategy 

Statistical Independence. We made several decisions 
during the literature review process based on commonly used 
meta-analytic literature review guidelines to reduce statistical 
dependencies (Cooper Hedges, 1996). Because samples are 
often reported across more than one study, we reviewed each 
of the studies to ensure the independence of the samples. One 
study (Glassberg, Hooper, & Mattison, 1999) was eliminated 
because it used a sample that had been previously reported. 
In the end, each of the studies represented an independent 
sample. 

Homogeneity Statistics. Hedges and Olkin’s (1985) 
categorical fixed-effects model was used to evaluate which 
study characteristics, if any, were moderators of effect sizes. 
Using this approach, we selected a priori variables that were 
thought to be significant moderators of effect sizes and grouped 
the studies respectively (Durlak & Wells, 1998). 

To determine whether the whole sample of effect sizes 
were homogeneous or heterogeneous, the homogeneity statis- 
tic Qj was calculated (Hedges Olkin, 1985). A significant 
Q-y would indicate that the effect sizes are not homogeneous 
across all the studies. This finding would suggest that the over- 
all sample of effect sizes can be further partitioned into smaller 
groups according to a priori categorical dimensions. After- 
wards, an omnibus between-class fit statistic (Q^) and an om- 
nibus within-class statistic (Qw) applied to the groupings 
(see Note). 

The Qb statistic tests whether the average effect sizes of 
each of the groupings are significantly different from each 
other, whereas the statistic tests for homogeneity of the 
effect sizes within each class. The best-case scenario is to ob- 
tain a 2 b statistic that is significant and a 2w statistic that is 
not significant (Durlak & Wells, 1998). In the event that the 
2w statistic is statistically significant (indicating that the ef- 
fect sizes within classes are not homogeneous). Hedges and 
Olkin (1985) suggested that it may be informative to partition 
the within-class fit statistic 2w i'^to individual study charac- 
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teristics. In so doing, subgroupings with especially poor fits 
(i.e., effect sizes that are heterogeneous) are revealed, as is in- 
formation regarding the possibility of other potentially more 
informative classification dimensions. 

Estimation of Grand Effect Size. MetaWin 2. 1 (Ros- 
enberg, Adams, & Gurevitch, 2000) was used to assess the 
grand effect. MetaWin 2.1 allows researchers to summarize 
the results of multiple studies using meta-analytic procedures. 
In this meta-analysis, we included a number of studies that 
compared a group of children with EBD to the normative sam- 
ple of a standardized instrument. In these cases, we set the 
control group size equal to the size of the EBD group to en- 
sure equal weighting. Because most studies compared the EBD 
group to a normative group, it was decided that a pooled stan- 
dard deviation would be appropriate. A pooled standard devi- 
ation also provides a better estimate of the population standard 
deviation (Rosenthal, 1996). Moreover, each ES was weighted 
by the inverse of its variance, a procedure that gives more 
weight to effect sizes emerging from studies with larger sam- 
ples (Durlak & Wells, 1998; Hedges & Olkln, 1985). Each ef- 
fect size comparison between the EBD group and a normative 
or comparison group was expressed in terms of the standard- 
ized difference between mean scores divided by the average 
population standard deviation (Rosenthal, 1995). The formula 
used to derive effect sizes was defined as follows: 

ES = (/Hgp - /Hc)/5DpQQ[gj 

Thus, a negative ES indicated that the EBD group was lower in 
achievement than was the control group or normative sample. 

Moderator Effect Sizes. Each study was coded accord- 
ing to the domains of interest to assess whether there were 
meaningful effect size differences across these dimensions. 
We examined the following variables for potential modera- 
tors: (a) subject area (i.e., reading, math, spelling, or written 
expression), (b) academic setting (i.e., not reported, general ed- 
ucation, resource room, self-contained room, separate school, 
residential treatment, home school, university/clinic, other, or 
combined settings), (c) age (i.e., under 12 years or 12 or more 
years), and (d) method of identification (i.e., school identified, 
DSM identified, or rating scale identified). 

Coding Agreement. Three graduate students reviewed 
and coded each of the final 25 studies. Interrater agreement 
was assessed for 32% (« = 8) of the articles. To assess inter- 
rater agreement, two reviewers read and coded the same arti- 
cles; their responses for each item were then compared, to 
determine agreement. In instances where coders disagreed, a 
third reviewer read and coded the article to determine congru- 
ence. Percentage of agreement was determined by dividing 
the number of agreements by the number of agreements plus 
disagreements and then multiplying by 100. Agreement ranged 
from 85% to 100%, with a mean of 98%. 


Supplemental Data Analysis 

Confidence Intervals. We calculated bootstrap confi- 
dence intervals around the grand mean and subordinate effects 
sizes. The bootstrap method was selected for the following rea- 
sons. Eirst, resampling methods such as bootstrap can be used 
to estimate confidence intervals because they generate their 
own distributions. Using this method allowed freedom from 
the distribution assumptions of parametric tests (Rosenberg, 
Adams, & Gurevitch, 2000). Second, the bootstrap method is 
more conservative. This is important because of the use of 
normative comparisons, which may overstate the difference 
between groups; thus, a conservative method seemed appro- 
priate. Third, because of the limited number of studies, there 
was a possibility of a violation of distributional assumptions. 
The use of bootstrap, which is a nonparametric test, avoids 
this problem. Interpretation of confidence intervals is straight- 
forward. In the case of individual comparisons, means whose 
bootstrap confidence intervals do not contain zero differ sig- 
nificantly from zero at the .05 level (Cooper & Hedges, 1996; 
Durlak & Wells, 1998). This indicates that the average effect 
size of the control (or normative) group differs significantly 
from that of the EBD group. In the case of comparisons across 
multiple areas, means whose confidence intervals do not over- 
lap indicate that these effect size means differ from each other 
significantly. 

Results 

Table 1 shows studies included in this analysis. The 25 studies 
included 2,486 participants with EBD. The mean age and IQ 
for the EBD sample was reported in 17 studies. The weighted 
mean age across those studies was 1 1 .22 years, and the weighted 
mean IQ was 94.89. The gender of the EBD participants was 
reported on 1,809 (82%) of the participants in 15 (71%) stud- 
ies. Of those participants, 80% were boys and 20% were girls. 
Demographic information pertaining to race was available 
for 1,330 (61%) of the participants with EBD and was re- 
ported in 10 (48%) studies. Of these participants, 69% were 
Caucasian, 27% were African American, 3% were Hispanic, 
and 1% were of mixed backgrounds. Because we were unable 
to disaggregate the academic results based on race and gen- 
der, we could not analyze these factors. Due to the possibil- 
ity that there were systematic differences between effect sizes 
derived from control groups (« = 49) versus those derived from 
normative comparisons (n - 52), we compared the results of 
the effect sizes derived from normative comparisons with those 
derived from control groups. Differences were not significant, 
so the two groups were combined for further analysis. We also 
tested for the possibility that there would be systematic dif- 
ferences across tests (e.g., effect sizes from the Wide Range 
Achievement Test might be higher than those of the Woodcock- 
Johnson 111 Tests of Achievement). With two exceptions, the 
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TABLE 1. Studies Included in Analysis 





Authors 

Participants/ 

identification 

Student 

characteristics 

Setting 

Academic areas 
assessed 

Dependent 

measures 

Anderson, Kutash, 

& Duchnowski 
(2001) 

N = 42, 36 boys, 

6 girls 

school identified 

Age = 5.90 yrs, 

IQ = 95.3 

Special education 
resource, separate day 
school or clinic 

Reading, arithmetic 

KTEA, FIAT, WJ 

Browne, Stotsky, & 
Eichorn (1977) 

A = 510, 428 boys, 

82 girls 

DSM identified 

Age = 12.50 yrs, 

IQ = 92.0 

Self-contained, 
separate day school, 
residential 

Reading, arithmetic, 
and written 
expression 

WRAT 

Coutinho (1986) 

A = 45 

rating scale 
identified 

Age = third grade 
to ninth grade at 
time of testing, 

IQ not reported 

NA 

Reading and 
vocabulary 

SAT, ITBS 

Duncan, Forness, 

& Hartsough (1995) 

A = 85, 66 boys, 

19 girls 

DSM identified 

Age = 12.80 yrs, 

IQ = 94.0 

Separate day school 

Reading, arithmetic, 
and written 
expression 

WJ 

Foley & Epstein 
(1992) 

A = 86, 78 boys, 

8 girls 

school identified 

Age = 13.30 yrs, 

IQ = 98.4 

Self-contained 

Reading, arithmetic, 
and written 
expression 

QSAT 

Fomess, Youpa, 
Hanna, Cantwell, & 
Swanson (1992) 

A = 41, 41 boys 

DSM identified 

Age = 9.10 yrs, 

IQ = 105.0 

Clinical outpatient 

Reading and 
arithmetic 

WRMT, FIAT, 
KMDT, Gray Oral 

Glassberg (1994) 

A = 252, 217 boys, 

38 girls 

school identified 

Age = 10.00 yrs, 

IQ = 100.0 

Resource room, self- 
contained, separate 
day school 

Reading, arithmetic, 
and written 
expression 

WJ 

Gresham, 

MacMillan, Bocian, 
Ward, & Forness 
(1998) 

A = 130, group only 
for gender 

rating scale 
identified 

Age = third grade, 
IQ = not reported 

General education 

Reading and 
arithmetic 

CTBS, MAT, ITBS 

Grizenko, Papineau, 

& Sayegh (1993) 

A = 30, 27 boys, 

3 girls 

DSM identified 

Age = 9.00 yrs, 

IQ = 100 

Separate day school 

Reading, arithmetic, 
and written 
expression 

WRAT 

Grizenko & Sayegh 
(1990) 

A = 23 

DSM identified 

Age = 9.00 yrs, 

IQ = not reported 

Residential 

Reading, arithmetic, 
and spelling 

WRAT 

Hodges & Plow 
(1990) 

A = 76, 50 boys, 

26 girls 

DSM identified 

Age = 10.00 yrs, 

IQ = 97.5 

University, psychi- 
atric facility, 
hospital 

Reading, arithmetic, 
and written 
expression 

WJ 

Lee, Elliott, & 

Barbour (1994) 

A = 58, all boys 

school identified 

Age = 14.40 yrs, 

IQ = 93.8 

General education, 
residential 

Reading, arithmetic, 
and written 
expression 

WJ, FIAT, WRAT 

Lopez, Forness, 
MacMillian, Bocian, 

& Gresham (1996) 

A = 48, group only 
for gender 

school identified 

Age = 7-12 yrs, 

IQ = not reported 

General education 

Reading, arithmetic, 
and written 
expression 

WRAT 

Mattison et al. 

(1986) 

A= 109 

school identified 

Age = 11.40 yrs, 

IQ = 95 

NA 

Reading 

WRAT 

(table continues) 
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(Table 1 continued) 

Authors 

Participants/ 

identification 

Student 

characteristics 

Setting 

Academic areas 
assessed 

Dependent 

measures 

Mattison, Spitznagel, 

& Felix (1998) 

V= 111 

school identified 

Age = 12.10 yrs, 

IQ = 93 

Self-contained, 
special school 

Reading 

WRAT 

Meadows, Neel, 

Scott, & Parker 
(1994) 

/V = 19, all boys 

school identified 

Age = 13.40 yrs, 

IQ = 90 

Self-contained 

Reading, arithmetic, 
and written 
expression 

WJ 

Naylor, Staskowski, 
Kenney, & King 
(1994) 

N = 54, 20 boys, 

34 girls 

DSM identified 

Age = 14.00 yrs, 

IQ = not reported 

University, psychi- 
atric facility, 
hospital 

Reading, arithmetic, 
and written 
expression 

WJ 

O’Brien et al. (1992) 

N = 35, 29 boys, 

6 girls 

school identified 

Age = 9.30 yrs, 

IQ = 98.8 

University, psychi- 
atric facility, 
hospital 

Reading, arithmetic, 
and written 
expression 

WRAT 

Rennick, Klinge, 

Hart, & Lennox 
(1978) 

V = 14, gender not 
reported 

DSM identified 

Age = 8.93 yrs, 

IQ = 95.6 

Not identified 

Reading, arithmetic, 
written expression, 
and other 

FIAT 

Richmond & Blagg 
(1985) 

N = 30, gender not 
reported 

school identified 

Age = 7-9, 

IQ = not reported 

Resource room 

Reading and 
arithmetic 

WRAT 

Rosenblatt & 
Rosenblatt (1999) 

N = 61, 52 boys, 

9 girls 

school identified 

Age = 11.80 yrs, 

IQ = not reported 

Separate day school 

Reading, arithmetic, 
and written 
expression 

WRAT 

Sinclair, Fomess, 

& Alexson (1985) 

N = 350, 254 hoys, 
96 girls 

DSM identified 

Age = 10.20 yrs, 

IQ = 94.0 

General education, 
resource room, self- 
contained, separate 
day school, university, 
psychiatric facility, 
hospital 

Reading and 
arithmetic 

FIAT WRAT 

Tramontana, Hooper, 
Curley, & Nardolillo 
(1990) 

N = 50, 40 hoys, 

10 girls 

DSM identified 

Age = 12.60 yrs, 

IQ = 94.3 

University, psychi- 
atric facility, 
hospital 

Reading, arithmetic, 
and other 

WJ 

Wilson, Cone, 

Bradley, & Reese 
(1986) 

N = 140, gender 
ratio not reported 

school identified 

Age = 11.89 yrs 

IQ = 93.2 

Separate day school 

Reading, arithmetic, 
and written 
expression 

WRAT 

Zimet & Farley 
(1993) 

N = 87, 61 hoys, 

26 girls 

DSM identified 

Age = 9.33 yrs. 

University, psychi- 
atric facility, 
hospital 

Reading, arithmetic, 
and written 
expression 

WRAT 


Note. KTEA = Kaufman Test of Educational Achievement (Kaufman & Kaufman, 1998); FIAT = Peabody Individual Achievement Test— Revised (Markwardt, 1998); WJ = Woodcock- 
Johnson III Tests of Achievement-Revised (Woodcock et al., 2001); DSM = Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2000); 
WRAT = Wide Range Achievement Test (Wilkinson, 1993); SAT = Stanford Achievement Test (1995); ITBS = low'a Test of Basic Skills (Hoover et al., 1996); QSAT = Quick-Score 
Achievement Test (Hammill et al., 1987); WRMT = Woodcock Reading Mastery Tests (Woodcock, 1998); KMDT = Key Math: Diagnostic Arithmetic Test (Connoly et al., 1971); 
Gray Oral = Gray Oral Reading Tests-Fourth Edition (Wiederholt & Bryant, 2001); CTBS = Comprehensive Test of Basic Skills (McGraw-Hill, 1990); MAT = Metropolitan 
Achievement Test (Harcourt Educational, 2001). 


95% bootstrap confidence interval for all tests overlapped, in- 
dicating no significant differences across instruments. The ex- 
ceptions were Anderson, Kutash, and Duchnowski (2001), with 
2 effect sizes using the Kauffman Test of Educational Achieve- 
ment, and Coutinho (1986) with 12 effect sizes using the Stan- 


ford Achievement Test. Because both exceptions were within 
single studies, there did not appear to he any significant ef- 
fects due to instrumentation. 

The 25-study database yielded 101 effect sizes. The 
weighted mean effect size for the sample was -0.6905 {SD = 


136 THE JOURNAL OF SPECIAL EDUCATION VOL. 38/NO. 3/2004 


.40), with a range of -3.371 to +0.503. According to Cohen 
(1988), an effect size of this magnitude denotes a moderate to 
large difference in the academic performance of students with 
EBD, compared to that of same-age peers without disabilities. 
Figure 1 illustrates the distribution of effect sizes. With the 
exception of one outlier, the distribution is essentially normal. 
Of the 101 effect sizes, 90 were negative, indicating that in 
89% of the comparisons, students with EBD performed lower 
In academics than did nondisabled control or norm groups. 
The grand effect size was also significantly different from zero 
(95% bootstrap confidence interval = -.8 1 to -.57). Thus, over- 
all academic performance was significantly lower for the EBD 
group. Finally, the overall homogeneity statistic was signifi- 
cant (Qj- = 704.96, p < .05) denoting the possibility of mod- 
erator variables. 

Moderators 

Table 2 provides data on the number of effect sizes for each 
moderator (k), weighted mean effect size for each moderator 
(d^ ), confidence intervals, and (which indicates whether 
or not effect sizes within group were homogenous). These 
analyses were conducted using procedures recommended by 
Hedges and Olkin (1985) and Rosenburg et al. (2000) and 
computed using MetaWin. According to Durlak and Wells 
(1998), a well specified fixed-effects model is reflected by a 
significant Qg, but a nonsignificant for each class within 
moderator. 


Academic Subjects. Academic subject areas possessed 
sufficient heterogeneity (Qg = 46.64, p < .05) among effect 
sizes to justify subcategories (e.g., reading, math). Overall, 
students with EBD performed significantly lower than the 
nondisabled or norm groups across all subjects and all sub- 
ject area categories were statistically different from zero. The 
largest effect sizes were obtained in mathematics (-.81) and 
spelling (-.81). The bootstrap confidence intervals for all of 
the academic subjects overlapped, denoting no statistical dif- 
ference in the academic performance of students with EBD 
between and among subject areas. The within-group homogen- 
eity statistics indicated, however, that the subject areas were 
not homogeneous. This finding indicated that heterogeneity 
greater than chance remained at the designated within-group 
subject category. Thus, subject matter as categorized did not 
explain all the variability in effect sizes. 

Setting. As with the academic subjects, setting con- 
tained sufficient variability among effect sizes to justify sub- 
categories (2 b = 150.71,/; < .05). All placement settings were 
statistically different from zero, which would suggest that 
students with EBD performed significantly lower than com- 
parisons, regardless of setting. The largest effect sizes were 
demonstrated by students in residential facilities demonstrated 
the largest (-1.49), followed by those who did not report this 
information (-1.04). The smallest effect sizes were found in 
resource rooms (-.33). With the exception of residential and 
self-contained settings, the within-group homogeneity statis- 
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FIGURE 1. Distribution of effect sizes. 
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tics were significant, indicating that — with the exception of 
residential and self-contained settings — setting as categorized 
did not explain all the variability in effect sizes. However, the 
residential setting only encompassed one study, so this should 
be viewed with caution as the results are likely due to the fact 
that effect sizes were derived from a single sample. 

Also, with the exception of the self-contained/university 
clinic comparison, bootstrap confidence intervals among the 
remaining placement comparisons overlapped, denoting no 
statistical difference between and among them. The bootstrap 
confidence interval for self-contained and residential students 
did not overlap with any other setting, indicating that students 
with EBD in self-contained classrooms and residential set- 
tings performed significantly lower in academics than did stu- 
dents in other settings. 


Age. To test the effect of age on academic performance, 
the participants from each study were assigned to one of two 
groups, based on whether the reported mean age for each sam- 
ple was 12 years and older or below 12 years. The two age 
groups demonstrated sufficient variability (Qg = 38.88, p < 
.05) among effect sizes to justify the use of subcategories (i.e., 
12 years or older, younger than 12 years). Both older and 
younger students with EBD were statistically significant from 
zero, which suggests that students with EBD performed sig- 
nificantly lower in academics than did the same age nondis- 
abled control or norm groups. As in subject area and setting, 
both age categories possessed within-group effect size vari- 
ability beyond chance, indicating that age as categorized did 
not explain all the variability in effect sizes. Bootstrap confi- 
dence intervals for both age groupings overlapped, indicating 


TABLE 2. Summary of Effect Size Moderator Statistics 


Characteristic 

k 

d. 

95% bootstrap Cl 

Qwi 

Omnibus 

Subject area 







Reading 

48 

-0.61 

-0.79 to -0.49 

357.07* 



Math 

30 

-0.81 

-0.99 to -0.57 

167.13* 



Spelling 

12 

-0.81 

-1.05 to -0.39 

86.63* 



Written expression 

11 

-0.46 

-0.79 to -0.24 

47.48* 

Gw 

658.32* 






Gb 

46.64* 

Setting 







General education 

10 

-0.54 

-0.80 to -0.29 

48.80* 



Resource room 

5 

-0.33 

-0.77 to -0.21 

70.40* 



Self-contained 

12 

-0.83 

-0.92 to -0.76 

4.15 



Separate school 

18 

-0.62 

-0.89 to -0.46 

65.87* 



University clinic 

22 

-0.47 

-0.66 to -0.28 

92.47* 



Combined 

8 

-0.76 

-1.04 to -0.44 

160.34* 



Residential 

3 

-1.49 

-1.64 to -1.29 

.60 



Not reported 

23 

-1.04 

-1.29 to -0.93 

111.65* 

Gw 

554.26* 






Gb 

150.71* 

Age 







12 years or more 

34 

-0.82 

-1.00 to -0.60 

257.69* 



Less than 12 years 

67 

-0.60 

-0.74 to -0.48 

408.39* 

Gw 

666.08* 






Gb 

38.88* 

Method of identification 







School 

42 

-A1 

-0.58 to -0.31 

220.21* 



DSM 

39 

-.74 

-0.95 to -0.57 

279.89* 



Rating scale 

20 

-1.12 

-1.48 to -0.94 

63.98* 

Gw 

564.08* 






Gb 

140.88* 


Note, k = number of effect sizes; = mean effect sizes; Cl = confidence intei'val; = omnibus within-class fit statistic; = omnibus within-class fit statistic; (2 b “ 
omnibus between-class fit statistic; DSM = Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 2000). 
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no statistical difference between ages in the performance of 
students with EBD. 

Method of Identification. To test for effects due to 
method of identification, we employed four categories based 
on identification methods in the studies: school identified, 
D5/l/-identified, rating scale-identified, described as receiving 
services for EBD. The diagnostic groups demonstrated suffi- 
cient variability (Qg = 140.88, p < .05) among effect sizes to 
justify the use of subcategories. Students in all three cate- 
gories were statistically significant from zero, which suggests 
that students with EBD in all categories performed signifi- 
cantly lower in academics than did the same age nondisabled 
control or norm groups. All three categories possessed within- 
group effect size variability beyond chance, indicating that 
identification method as categorized did not explain all the 
variability in effect sizes. Bootstrap confidence intervals for 
school-identified and D5/lf-identified groupings overlapped, 
indicating no statistical difference in the performance of these 
students. Rating scale-identified students differed signifi- 
cantly from school-identified, but not D5/lf -identified, students. 
However, because the rating scale-identified groups were in- 
cluded in only two studies, some caution is in order when gen- 
eralizing these results. 

Discussion 

The purpose of this investigation was to quantitatively synthe- 
size the research on the academic status of students with EBD, 
to determine the magnitude of difference in performance when 
compared to students without disabilities. In this section we 
discuss participant characteristics and problems with gener- 
alization, the overall level of academic deficits of children 
with EBD, academic deficits across subject areas and settings, 
and achievement deficits across age groups. 

Participant Characteristics 

Problems with thorough reporting of characteristics of sam- 
ples have been noted in previous reviews of academic status 
of children with EBD (e.g., Mooney et al., 2003; Trout et al., 
2003). Although more than 2,000 students were represented 
in this study, it is difficult to assess how representative these 
students were of the total EBD population. Race, gender, and 
SES can be potentially important moderators; however, the 
extent to which these data were reported was less than opti- 
mal. Nearly 30% of studies did not provide information on 
the gender of participants, and no study provided disaggre- 
gated data on female participants. As a result, we could not 
assess whether gender was a significant moderator of achieve- 
ment. This is a serious gap in our knowledge base, because at 
present, approximately 20% of children with EBD are girls 
(U.S. Department of Education, 2001). Data on race and eth- 
nicity were reported in fewer than half the studies. Again, this 


represents a serious omission. Minority groups make up more 
than 25% of the U.S. population (U.S. Census Bureau, n.d.). 
African Americans account for slightly more than 12% of the 
U.S. population, and between 1990 and 2000, the percentage 
of Hispanics in the United States doubled (U.S. Census Bu- 
reau, n.d.). These groups also constitute a significant propor- 
tion of the EBD population. African American and Hispanic 
groups constitute 27.3% and 8.9% of the EBD population in 
the United States, respectively (U.S. Department of Educa- 
tion, 2001). The lack of information on SES is also troubling. 
Knowledge of SES is important because low SES is a well- 
recognized risk factor for EBD (Cullinan, 2002). Another 
problem lies in the nature of the EBD groups sampled. Most 
of the studies appear to have reported convenience samples. 
None of the studies used epidemiological methods. Thus, we 
cannot be certain to what extent the students included in this 
analysis are representative of the current EBD population. 

Overall Achievement 

The findings demonstrate that there is a moderate to large 
(-.69) overall difference in the academic performance of stu- 
dents with EBD, compared to students without disabilities. 
This is consistent with previous research which has noted per- 
sistent problems with low achievement (e.g., Kauffman, 2001; 
Kauffman et al., 1987). The use of effect sizes allow for an 
estimate of the proportion of children in the EBD group who 
score below the mean of contrast group. In this case, 75% of 
students in the EBD group scored below the mean of the con- 
trast group. The overall mean achievement level of the EBD 
group is at the 25th percentile. 

Interpretation of the overall degree of academic deficits 
among children with EBD should be done in light of two fac- 
tors. First, and most important, we have no information on the 
exact timing of the academic assessments or on instructional 
programming. There could well be significant differences in 
the degree of academic deficits between children whose data 
were taken at intake and those who had been previously iden- 
tified and provided with effective instructional programming 
for a period of time. Hopefully, the degree of deficit would 
be less if an effective instructional program was in place. Sec- 
ond, we have no information on age of identification. It is 
possible that there would be significant differences between 
children who were identified and treated early. 

Academic Subject Area 

Students with EBD performed significantly below their peers 
in all academic subject areas. Though problems with reading 
and literacy have been the focus of public concerns and re- 
search, the two areas with the greatest absolute deficits were 
math and spelling; however, differences across the academic 
areas were not significant. This was somewhat surprising. One 
possible explanation for the lack of difference is that it reflects 
the increased emphasis now placed on reading and written ex- 
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pression. This does not appear to be the case, because studies 
reporting reading data were evenly distributed over the time 
span reviewed, so factors related to increased or improved 
reading instruction are unlikely to have resulted in the ob- 
served differences. It is also possible that placement differ- 
ences or age differences may have affected results. Because 
of the possibility that differences across academic subject 
areas could result from systematic differences across place- 
ment and subject area (e.g., if there were more math effect 
sizes from self-contained settings or from older students), we 
conducted a post hoc chi-square analysis. Results of this analy- 
sis were not significant for academic subject by setting, X^(21 , 
N = 80) = 30.97, p = .07. Thus, differences in settings would 
not account for the lack of differences across subject areas. 
Age by subject area results were significant, %^(3, N = 101) 
= 8.12, p = .043. However, this was due to a disparity within 
one area, spelling, for which younger children had 1 1 effect 
sizes, compared to a single effect size for older children. An- 
other possible explanation is that inadequate math skills are 
more influenced by limited classroom exposure than reading 
(Forness, Bennett, Tose, 1983; Naylor, Staskowski, Kenney, 
& King, 1994). 

The size of the deficit in spelling was somewhat surpris- 
ing because research has suggested that there is a strong re- 
lation between reading and spelling; correlations between 
reading and spelling achievement range from .50 to .90 (Gra- 
ham, Harris, & Chorzempa, 2002). However, Treiman (1993) 
noted that some children who experience difficulty spelling have 
little difficulty with reading. Age of participants is also a po- 
tential factor. Of the 12 spelling effect sizes, 11 were from the 
group of children younger than 12 years. There is some evi- 
dence that spelling develops relatively slowly in low-achieving 
students (O’Malley, Francis, Foorman, Fletcher, & Swank, 
2002). Additionally, deficits in spelling may be explained by 
limited short-term memory span common in children with 
attention and behavior problems (Levy & Hobbes, 1989). 
However, the fact that the Q statistic was significant indicates 
that subject areas alone do not explain all the observed vari- 
ability. 

It is interesting to note that the observed deficits in 
achievement in the present study were consistent with those 
of reported by Kavale and Nye (1985-1986) for children with 
learning disabilities (LD). Kavale and Nye found overall dif- 
ferences in academic achievement of .683 between a control 
group and children with LD. Across subject areas, differences 
between the children with LD and the control group for com- 
parable academic subject areas were as follows: reading, .757; 
math, .607; spelling, .726. This suggests that the major dif- 
ference between LD and EBD groups may lie in reading and 
that there may be little difference between LD and EBD in 
terms of math and spelling achievement. This is consistent 
with current conceptualization of LD — that reading is the core 
deficit area for children with LD, due to an underlying deficit 
in phonological processing (Adams, 1996; National Research 
Council, 1998). Thus, children with EBD may have a slightly 


different profile from children with LD in that they are some- 
what less likely to demonstrate underachievement in reading. 

Setting Differences 

Across general education, resource room, self-contained, and 
special school settings, there was no significant difference in 
academic performance. This finding was surprising. One might 
expect that students placed in general education would have 
higher academic performance than students in resource rooms 
or more restrictive setting. However, this does not appear to 
be the case. We would caution, however, that the lack of dif- 
ference may be due to the variability demonstrated within each 
setting. 

For students in self-contained and residential settings, 
the 2 yv statistic was not significant, suggesting that no other 
moderators were necessary to explain the observed variabil- 
ity. We would caution that although the data suggest homo- 
geneous effect sizes, we cannot determine exactly why this 
occurred. There are three plausible explanations for the ob- 
served result. First, the residential setting comprised only one 
study with a small sample (« = 23). Thus, for this group, the 
results may be a statistical artifact. Second, it is possible that 
students in self-contained settings and residential settings are 
homogeneous. We would expect that students in self-contained 
and residential settings would generally demonstrate more 
serious behavior problems. The onset, frequency, and per- 
sistence of behavior problems in children is related to poor 
academic performance (McEvoy & Welker, 2000). Thus, it is 
possible that the students in these settings are a more homo- 
geneous group in terms of academic deficits. This could be 
seen as supporting the need for self-contained placement op- 
tions. A third explanation is that the lack of significant varia- 
tion among self-contained and residential groups is due to the 
setting. The students in these settings may not receive con- 
sistent, effective academic instruction, perhaps because of a 
focus on behavior management strategies. From this perspec- 
tive, academic deficits are the result of placement in a setting 
that produces lower academic performance. Again, we caution 
that the lack of information on programming precludes any 
firm conclusions. It is also important to note that some effect 
sizes reflected multiple settings that included self-contained 
groups and that there were no significant differences across 
general education, resource, and self-contained settings. 

Age Differences 

The difference in academic achievement across age groups 
was not statistically significant, and the overlap of confidence 
intervals across the groups was pronounced. This was surpris- 
ing, as previous research has suggested that underachievement 
tends to increase with age (Coutinho, 1986). And, once again, 
the significant statistic indicates that age alone is insuffi- 
cient to explain observed variability. There is little longitudi- 
nal data on the academic status of children with EBD. Only 
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one study reported longitudinal data using standard scores 
(Anderson et al., 2001), and this study did not find significant 
decreases over a 4-year time span. One explanation for the lack 
of difference across age groups could be due to the fact that 
many previous studies have reported deficits in terms of grade- 
equivalent scores. Coutinho’s study is often cited as evidence 
that academic deficits increase over time. However, Coutinho 
based this conclusion on the increase in grade-equivalent dif- 
ferences over time and did not actually test for differences. 
The problems inherent in grade-equivalent scores are well 
documented (e.g., Linn & Gronlund, 2000). Grade-equivalent 
scores are less reliable with older children, are not directly 
comparable across age groups, and can decline independent 
of any real change in academic status (Venn, 2004). For ex- 
ample, a student who is 1 year below grade level at age 6 and 
a student who is 3 years below grade level at age 16 may have 
equal deficits in terms of their rank in the population. This 
could explain why our results differed from Coutinho’s. It is 
important to note that in our study, groups’ age scores were 
based on the mean reported age and that the comparisons 
made are not within-group. Rather, they are across groups that 
may differ across potentially important variables (e.g., SES, 
gender) that were not assessed in this analysis. 

Method of Identification 

Students across all three methods of identification demon- 
strated significant deficits in achievement. There was a sig- 
nificant difference between school-identified and rating 
scale-identified students, which suggests that academic status 
differs across diagnostic methods. This should be viewed with 
caution, however, because only two studies used rating scale 
identification. There was no significant difference across the 
two largest groups — school-identified and D5/lf -identified stu- 
dents. This was surprising considering that many of the DSM- 
identified groups were in hospitals or psychiatric facilities. 
One might expect that students in hospitals or psychiatric fa- 
cilities would demonstrate concomitant levels of impairment 
in academics. Several factors temper interpretation of this re- 
sult, however. First, the confidence interval overlap across the 
two groups was minimal. Second, we used very conservative 
methods to assess significance. Finally, the within-group Q 
statistic was significant for all three groups, indicating that 
identification method does not explain all the variability in ef- 
fect sizes. 

Limitations 

Despite an exhaustive literature search, we were able to lo- 
cate only 25 studies that met criteria for the meta-analysis. A 
more refined picture may be possible with more studies. The 
fact that we were able to locate only 25 studies demonstrates 
the need for more research in this area. Including studies that 
used curricular-based measures would have increased the num- 
ber of studies slightly; however, our concern was overall aca- 


demic status. We did not think it appropriate to equate the 
standardized and curriculum-based measures. Given the small 
number of studies, several caveats common to meta-analytic 
studies should be considered when interpreting the results. 
First, only one within-group statistic was significant; thus, 
additional moderators or different categorizations or combi- 
nations of moderators are necessary to explain the variability 
present. Second, given the ratio of effect sizes to studies (i.e., 
101 :25), statistical dependencies could have accounted in part 
for the findings, because multiple effect sizes were calculated 
for the same participants. Third, because each effect size was 
treated as an individual unit, the results may be skewed, as 
studies with larger sizes were given greater weight. Fourth, due 
to the small number of effect sizes for some moderators (e.g., 
spelling or resource room setting), we need to interpret these 
results with great caution. Fifth, the participants included were 
likely highly heterogeneous, due to such factors as variation 
in definitions of FED across states. This likely served to in- 
crease variance across studies. Finally, all the studies reported 
were cross-sectional “snapshots” of academic performance. 
This study cannot address within-student changes over time. 

Future Research 

Our findings highlight several areas in need of future research. 
First, the fact that we were able to locate only 25 studies with 
101 effect sizes that provided academic data indicates that 
additional research aimed at academic achievement among 
children with FED is necessary. A similar meta-analysis by 
Kavale and Nye (1985-1986) assessing the achievement of 
students with LD generated 260 effect sizes. Second, inade- 
quate demographic reporting limited our understanding of the 
academic characteristics of students with FED. This problem 
appears to be endemic. For example, in a review of 66 stud- 
ies assessing the academic status of students with FED, Trout 
and colleagues (2003) found that only 44% of the studies in- 
cluded demographic information pertaining to race, and few 
of those studies reported students’ academic performance 
across specific subcategories (i.e., gender, race, and age groups). 
Similar problems were noted by Mooney et al. (2003). Given 
the concerns with overidentification of minorities and the in- 
creasing identification of girls with FED, researchers need to 
provide detailed demographic information and disaggregated 
data that will allow analysis of subgroups. Third, researchers 
have practiced an overreliance on measuring academic per- 
formance by age- and grade-equivalent scores (e.g., Rosenblatt 
& Rosenblatt, 1999; Scruggs & Matropieri, 1986). Although 
age- and grade-equivalent scores may be easier to interpret, 
they do not provide information on whether the child’s per- 
formance is within average limits, allow for normative com- 
parisons, or allow for comparisons across studies. Finally, we 
looked at academic performance in a broad context. Future re- 
search is needed to examine the academic performance of stu- 
dents with FED within specific academic skill sets, such as 
reading comprehension and mathematical reasoning. 
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Implications 

The findings from our study suggest that students with EBD 
are in need of additional effective academic instruction across 
all subject areas, settings, and age groups. From a research 
perspective, these findings emphasize the need for additional 
research to examine causal factors related to academic under- 
achievement, academic performance within specific subject 
areas, and the long-term impact of underachievement in spe- 
cific subject areas. The relationship between placement and 
achievement and the effects of gender and ethnicity should 
also be scrutinized. Moreover, understudied academic areas, 
such as written language and content areas, should receive at- 
tention. Reporting standards for participants should also be 
improved, to allow for analysis of important moderator vari- 
ables. From a practitioner’s perspective, the findings from this 
study emphasize the need for teachers to continue to monitor 
and measure the academic performance of students with FBD 
and to use scientifically based academic interventions that ad- 
dress deficits across all academic subjects and settings. There 
is also a need to attend to math and spelling, which may be 
serious deficit areas. From an administrator’s perspective, the 
findings underscore the need to train teachers and parapro- 
fessionals to use scientifically based instructional programs to 
positively affect the academic performance of students with 
FBD. From a policy perspective, the findings from this study 
emphasize the need for policies that encourage the measure- 
ment of academic performance and outcomes at every stage 
of development to determine the effectiveness of interven- 
tions. 
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NOTE 

The fit statistics Qj, Qg, and analogous to the partitioning of 

the total variance of effects into the variance occurring between [Qg] 
and within [2w] groups such that Qt = Gb + Gw- 
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